Apparatus for playing back recorded video images related to event,and method thereof

ABSTRACT

An apparatus includes a recording unit configured to record a video image, a generation unit configured to generate, when a first type of event in which a state of an object changes is detected from the video image, an index associating the first type and a second type of event related to the first type in the video image, and a playback unit configured to play back, when the first type is specified, a video image concerning the second type corresponding to the specified first type based on the index.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an apparatus for playing back recordedvideo images related to an event and a method thereof.

2. Description of the Related Art

Installation of a digital monitoring system including a plurality ofnetwork cameras, a recording apparatus to record camera video images,and a viewer to play back live video images or recorded video images isproceeding at a rapid rate. Japanese Patent Application NationalPublication (Laid-Open) No. 2007-505523 discusses a method for enablinga corresponding event on a time line when a camera is specified toclarify the correspondence between a specific camera and an event. Also,a method for playing back recorded video images captured by thecorresponding camera from the corresponding time when an event isspecified is discussed. U.S. Pat. No. 6,144,375 discusses a method forconfiguring a three-dimensional model of scenes from video images takenfrom a plurality of cameras, establishing correspondences between eventsand video images, and displaying events for each object.

An event in which a person carrying an object appears during capturingof moving images and the person then goes away while leaving behind theobject can be detected as a desertion detection event. Here, if adesertion detection event occurs, it is useful if the person who hasbrought in the deserted object can be checked. However, according to aconventional technique, in order to play back video images of an objectrelated to an event, it is usual to sequentially specify related eventsafter the relationship between the object and events is considered bythe user. Also, according to a conventional technique, a windowdedicated to the object is separately displayed to know correspondencerelations between the object and events. Thus, it is difficult tointuitively understand the correspondence relations between the objectand events.

SUMMARY OF THE INVENTION

According to an aspect of the present invention, an apparatus includes arecording unit configured to record a video image, an index generationunit configured to generate, when a first type of event in which a stateof an object changes is detected from the video image, an indexassociating the first type and a second type of event related to thefirst type, and a playback unit configured to play back, when the firsttype in the video image is specified, a video image concerning thesecond type corresponding to the specified first type based on theindex.

Further features and aspects of the present invention will becomeapparent from the following detailed description of exemplaryembodiments with reference to the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate exemplary embodiments, features,and aspects of the invention and, together with the description, serveto explain the principles of the invention.

FIG. 1 is a diagram illustrating an example of a system configuration ofa network camera system according to an exemplary embodiment of thepresent invention.

FIG. 2 is a diagram illustrating an example of a hardware configurationof a playback apparatus.

FIG. 3 is a diagram illustrating an example of a display screen.

FIG. 4 is a diagram illustrating an example of event information.

FIG. 5 is a diagram illustrating a presence time of objects and an eventwhen desertion detection occurs.

FIGS. 6A and 6B are diagrams illustrating an example of generation of anindex by extracting video images before and after events related to thedesertion detection.

FIG. 7 is a diagram illustrating an example of index information.

FIG. 8 is a flow chart illustrating an example of a processing procedurefor one process by the playback apparatus.

FIG. 9 is a diagram illustrating a presence time of objects and eventswhen taking away detection occurs.

FIGS. 10A and 10B are diagrams illustrating an example of generation ofan index by extracting video images before and after events related tothe taking away detection.

FIG. 11 is a flow chart illustrating an example of a processingprocedure of one process by the playback apparatus.

FIG. 12 is a diagram illustrating an example of a correspondencerelation between an object region display and an event display on a timeline.

FIG. 13 is a diagram illustrating another example emphasizing eventsconcerning the same object on the time line.

FIG. 14 is a diagram illustrating an example of a screen of a recordingapplication when objects and events among a plurality of cameras aredisplayed by establishing correspondences therebetween.

FIG. 15 is a flow chart illustrating an example of a processingprocedure of one process by the playback apparatus.

DESCRIPTION OF THE EMBODIMENTS

Various exemplary embodiments, features, and aspects of the inventionwill be described in detail below with reference to the drawings.

In a first exemplary embodiment of the present invention, when an eventconcerning an object occurs, a playback apparatus for network camerasgenerates an index describing playback timing of events concerning anobject related to the object. When, for example, a desertion detectionevent occurs, the playback apparatus generates an index describing thedesertion detection event, an approximate time at which a person who hasbrought in a deserted object appears, and an approximate start time ofdesertion. Next, when a user specifies a symbol indicating a desertiondetection event on a time line where symbols indicating events aredisplayed at positions of their occurrence times, the playback apparatuscontinuously plays back recorded video images based on an indexincluding the desertion detection event. As a result, there is no needfor the user of a network camera system (or a playback apparatus) in thepresent exemplary embodiment to search for related events and manuallyplay back such related events, so that the user can immediately checkimportant video images.

A system configuration of a typical network camera system in the presentexemplary embodiment is illustrated in FIG. 1. In the network camerasystem, as illustrated in FIG. 1, a plurality of network cameras 101,102, and 103 and a playback apparatus 200 including a recordingapparatus 2100 are connected via a network. The network camera 101 has afunction to transmit video images via the network, a motion detectionfunction of video images, and a detection function of an object regionin a video image.

The playback apparatus 200 has a function to acquire a video image froma network camera, a function to save the acquired video image in therecording apparatus 2100, a function to play back the received/savedvideo images, a motion detection function of video images, and adetection function of an object region in a video image.

These functions of the playback apparatus 200 are realized by programsstored in a read-only memory (ROM) 207 or a hard disk drive (HDD) 209being executed by a central processing unit (CPU) 206.

FIG. 2 is a diagram illustrating an example of the hardwareconfiguration of the playback apparatus 200.

A display apparatus 201 displays, for example, windows, icons, messages,menus and other user interface information. A video random access memory(VRAM) 202 stores video images to be displayed on the display apparatus201. The display apparatus 201 may be hardware constituting the playbackapparatus 200 or an apparatus (hardware) that is connected to theplayback apparatus 200 and different from the playback apparatus 200. Inthe example in FIG. 2, the display apparatus 201 is different fromhardware constituting the playback apparatus 200.

Image data stored in the VRAM 202 is transferred to the displayapparatus 201 according to predetermined rules and video images arethereby displayed on the display apparatus 201. A compact disk drive(CDD) 203 is an apparatus that reads/writes various control programs anddata from/to recording media such as CD-ROM and CD-R. Instead of theCDD, a digital versatile disc (DVD) drive may be used.

A keyboard 204 has various keys to input characters. A pointing device(PD) 205 is used to point to icons, menus, or other objects displayed,for example, on a display screen of the display apparatus 201. The CPU206 controls each device based on a control program stored in the ROM207, the hard disk drive (HDD) 209, a flexible disk drive (FDD) 210, oran external storage apparatus (CDD) 203. As already described above, thefunctions of the playback apparatus 200 are realized by programs beingexecuted by the CPU 206 of the playback apparatus 200.

The read-only memory (ROM) 207 stores various control programs and data.A random access memory (RAM) 208 has a work area for the CPU 206, a datasave area during abnormality handling, a load area of control programsand the like.

The HDD 209 stores various control programs and various types of data. Anetwork interface (Net-I/F) 211 performs communication with otherinformation processing apparatuses, printers and the like via a network213. A CPU bus 212 includes an address bus, a data bus, and a controlbus. A control program can be provided to the CPU 206 from the ROM 207,the HDD 209, the FDD 210, or the CDD 203 or from other informationprocessing apparatuses via the network 213.

The function of the recording apparatus 2100 can be realized by externalstorage apparatuses such as a hard disk drive being controlled by theCPU 206 inside the playback apparatus 200. The playback apparatus 200performs communication with the network camera 101 via a network controlunit to acquire video images, which are saved in the recording apparatus2100. At this point, communication between the network camera 101 andthe playback apparatus 200 is performed based on, for example, HyperTextTransfer Protocol (HTTP), but the present exemplary embodiment is notlimited to a specific protocol.

Here, the motion detection function of video images of the playbackapparatus 200 calculates a quantity of motion in video images based on,for example, a difference between adjacent frames and, if the quantityof motion in video images exceeds a certain reference quantity, causes amotion detection event. The detection function of an object region in avideo image of the playback apparatus 200 detects a region of a certainpreset size or more obtained by performing region connection processingon, for example, a background difference image as an object region. Inthis case, a technique of improving detection accuracy by usingknowledge about objects obtained from object region detection results inthe past is generally used. The motion detection function of videoimages and the detection function of an object region may be provided onthe network camera 101 side or the playback apparatus 200 side. In thepresent exemplary embodiment, techniques of motion detection and objectregion detection are not to be limited.

Next, an example of the display screen of the playback apparatus 200 inthe present exemplary embodiment will be described with reference toFIG. 3. FIG. 3 is a diagram illustrating an example of the displayscreen. FIG. 3 illustrates a window in the screen and the windowincludes a window frame 300, image display regions 310, 320, and 330,and a timeline 340. Frames 311 and 312 indicating an object region aredisplayed in the image display region 310. In addition to theillustrated frame as a display example of the object region,representations of any shape such as a block and ellipse can be used.The time line 340 indicates recording states and occurrence times ofevents. More specifically, graphic symbols indicating events arearranged along the time axis in the time line 340.

The right end of the time line 340 indicates the current time and theplayback apparatus 200 displays display events so that, with the passageof time, display events shift from right to left. If the user specifiesthe current time or a past time, recorded video images from thecurrently enabled camera are played back from the specified time. Eventsinclude the activation/stop of a system, the start/stop of recording,changes of the input state of an external sensor, changes of the motiondetection state, the appearance/exit of an object, desertion, and takingaway. In FIG. 3, while an event 341 is represented as a rectangularform, other forms than the rectangular form can also be represented.

Next, an example of event information in the present exemplaryembodiment will be described with reference to FIG. 4. FIG. 4 is adiagram illustrating an example of event information. The playbackapparatus 200 manages events by assigning IDs in order of occurrencetime. Management content includes the occurrence date/time, type,priority, device ID, camera ID, object ID, and event details. The typeis grouping and the details are a more concrete description of an event.For desertion detection, for example, the type is “abnormalitydetection” and the details are “standing-still detection”. The device IDindicates an ID of a sensor or a camera device that became an eventsource. The priority is a priority concerning an event notification andthe display color and the alarm notification to the user are setseparately for each level of priority. The object ID is a serial numberassigned to each detected object individually.

Next, the method for generating an index in the present exemplaryembodiment will be described. In the present exemplary embodiment,desertion detection is taken as an example. FIG. 5 is a diagramillustrating a presence time of objects and an event when desertiondetection occurs. An object B is a bag and an object A is a person whocarries the bag. The object A appears at time 411, leaves behind theobject B at time 421, and exits at time 431. The object A appears attime 421 and is determined to be at rest for a predetermined time orlonger by the playback apparatus 200 at time 441. As a result, adesertion detection event occurs at time 441. At this point, the objectB is an object separated from the object A and thus, a relationshipbetween the objects A and B is defined by the playback apparatus 200.

Separation and integration of objects become possible by analyzingmotions of the objects. If, for example, a new object arises from thesame position as an existing object immediately after the size of theexisting object increases rapidly, the playback apparatus 200 candetermine that the existing object has separated. If, on the other hand,two existing objects or more are present at the same position and thenumber of apparent objects is one for a predetermined time or longerthereafter, the playback apparatus 200 can determine that the existingobjects have integrated.

Here, the playback apparatus 200 can improve detection accuracy ofseparation/integration of objects by using a human detection technique.The human detection technique is a technique to determine a human regionor a position by using body features or face features in a video image.When the human detection technique is applied to separation of objects,a rule is set down that while a person can separate into a person and anobject other than the person, a person cannot separate into a pluralityof persons. Conversely, when the human detection technique is applied tointegration of objects, a rule is set down that while a person and anobject can be integrated into a person, a plurality of persons cannot beintegrated into a person. Accordingly, an erroneous detection concerningseparation/integration by an overlapping operation of persons in theplayback apparatus 200 can be reduced.

Next, the generation method of an index when a desertion detectionoccurs will be described with reference to FIGS. 6A and 6B. FIGS. 6A and6B are diagrams illustrating an example of generation of an index byextracting video images before and after events related to the desertiondetection. FIG. 6A illustrates recorded video images. FIG. 6B is anexample of played back video images based on the index. In FIG. 6A, therespective events at the time 411 of the appearance of a person, thetime 421 of the start of desertion, the time 431 of the exit of theperson, and the time 441 of the detection of desertion described withreference to FIG. 5 are illustrated in the recorded video images 500.

When a desertion detection event 442 occurs, what the user wants to knowis a person who has left behind the deserted object and the moment ofthe desertion. Thus, as illustrated in FIG. 6B, the playback apparatus200 plays back video images before and after the time 411 of theappearance of a person and the time 421 of the start of desertioncontinuously. If the playback time before or after an event is set asΔT, video images 510 of the duration time 4ΔT will be played back by theplayback apparatus 200. Here, video images before and after thedesertion detection event 442 are changeless video images and thus, theplayback apparatus 200 omits the playback thereof. The playback timebefore or after an event is not limited to ΔT and may be set optionallyfor each event.

An example of information stored as an index is illustrated in FIG. 7.FIG. 7 is a diagram illustrating an example of index information storedby the playback apparatus 200 as an index. As illustrated in FIG. 7,index information includes three types of information: attention eventinformation, related event information, and operation information. Theattention event information is an event that causes generation of anindex and is, for example, the desertion detection event in FIG. 5. Anevent ID of the event is stored as the attention event information. Therelated event information is, for example, the time 421 of theappearance of an object corresponding to the attention event informationor the time 411 of the appearance of a separation source object of theobject. The event ID of the event is stored as the related eventinformation. However, a comment illustrated in FIG. 7 is not necessarilyneeded. The operation information is a list of the start time and stoptime of the playback. More specifically, the operation informationincludes operation content such as the start and end, camera ID, time,and related event IDs. As already described above, the playback time andstop time are determined by the playback apparatus 200 depending on theoccurrence times of related events.

Various techniques can be used for how to represent event information inFIG. 4 or the index in FIG. 7. For example, the event information or theindex can be represented in a list form of a structure in a program orsaved in a file or transmitted in the XML form.

The user causes the playback apparatus 200 to play back video images byspecifying a desertion detection event on the time line 340. Methods forspecifying an event include double-clicking and a method for using acontext menu, but the present exemplary embodiment does not limit themethod thereof.

Next, the processing procedure for one process by the playback apparatus200 in the present exemplary embodiment will be described with referenceto FIG. 8. FIG. 8 is a flow chart illustrating an example of aprocessing procedure for one process by the playback apparatus 200. Theprocess is a process of the graphical user interface (GUI) and morespecifically an event processing process. For simplification ofdescription, only events directly related to the present exemplaryembodiment are illustrated. As already described above, an eventprocessing process is executed by a program being executed by the CPU206.

In step S701, the CPU 206 performs initialization. In step S702, the CPU206 waits for an operational input from the user. When an operationevent occurs (YES in step S702), then in step S703, the CPU 206determines whether the event is a desertion/taking away detection event.If the operation event is a desertion/taking away detection event (YESin step S703), then in step S704, the CPU 206 generates an indexaccording to the technique described above.

If the operation event is other than a desertion/taking away detectionevent in step S703 (NO in step S703), then in step S710, the CPU 206determines whether the detection event is a playback instruction input.The playback instruction input is generated as a result of an event onthe time line being designated. If the detection event is the playbackinstruction input (YES in step S710), then in step S711, the CPU 206plays back the index of recorded video images related to the designatedevent. If the detection event is not the playback instruction input (NOin step S710), then in step S720, the CPU 206 performs other eventprocessing before returning.

According to the processing described above, when an event concerning anobject occurs, the playback apparatus 200 generates an index of videoimages of other objects related to the object and plays back videoimages according to the designation. More specifically, when a desertiondetection event occurs, the playback apparatus 200 generates an indexcapturing moments of the appearance of a person who has brought in adeserted object and the desertion of the object and plays back relatedrecorded video images based on the index according to the designation ofan event on the time line.

Next, a second exemplary embodiment of the present invention will bedescribed. There is a method for generating a video index when an eventconcerning an object occurs after waiting until all objects related tothe object exit. This method is effective if the index should include,when, for example, a taking away detection event occurs, up to the exitof a person who has taken away an object from the screen. On the otherhand, a delay may arise between the time when the taking away detectionevent occurs and the time when an index is generated and video imagesare played back.

The system configuration of the present exemplary embodiment is similarto that of the first exemplary embodiment and thus, a descriptionthereof will not be repeated and the description below focuses ondifferent portions from the first exemplary embodiment. The method forgenerating an index is illustrated in FIGS. 9 and 10.

FIG. 9 is a diagram illustrating the presence time of objects and eventswhen taking away detection occurs. An object A is a person and an objectB is a background region that appears after taking away occurs. Theobject B is the background region that newly appears after taking awayoccurs because when object region extraction using backgrounddifferences is used, the playback apparatus 200 stores scenes beforetaking away occurs as a background and thus, a region such as a wallthat appears after the taking away is first determined to be a newobject region. Events concerning the objects A include an appearance ofa person 811, a start of taking away 821, and an exit 831. The object Bappears after being separated from the object A at detection time of thestart of taking away 821, and a taking away detection event 841 occursbecause the object B is in a resting state for a predetermined time orlonger. FIG. 9 is different from FIG. 5 in that an event of the exit 831of the taking-away person occurs after the taking away detection event841 occurs. Here, the playback apparatus 200 can measure the fact thatthe object B has separated from the object A by using an identificationunit or method like the one described in the first exemplary embodiment.

FIGS. 10A and 10B are diagrams illustrating an example of the generationof the index by extracting video images before and after events relatedto the taking away detection. FIG. 10A illustrates recorded video imagesfrom the appearance of the object A to the exit of the object B andevents. FIG. 10B illustrates played back video images. In FIG. 10A, therecorded video images 910 include the appearance of a person 811, thestart of taking away 821, confirmation of the taking away detection (theexit 831 of the taking-away person), and the event of exit of the person(the taking away detection event 841). When the taking away detectionoccurs, it is necessary for the user to verify a taking-away act. Forthis purpose, the appearance of a person who has taken away an object,the moment of taking away the object, and the exit while carrying theobject are important moments.

Thus, as illustrate in FIG. 10B, the playback apparatus 200 in thepresent exemplary embodiment generates an index storing times of ΔTbefore or after the appearance of a person 811, the start of taking away821, and the exit of the person 842. As a result, video images 920 ofthe duration time 6ΔT will be played back. Like the first exemplaryembodiment, the playback time ΔT before or after an event may be setoptionally for each event.

Next, the processing procedure for one process by the playback apparatus200 in the present exemplary embodiment will be described with referenceto FIG. 11. FIG. 11 is a flow chart illustrating an example of theprocessing procedure of one process by the playback apparatus 200. Thisprocess is a process concerning the GUI and more specifically, an eventprocessing process. Like FIG. 8, for simplification of description, onlyevents directly related to the present exemplary embodiment areillustrated. A different portion from FIG. 8 will be described.

In step S1003, the CPU 206 determines whether the event is adesertion/taking away detection event. If the event is adesertion/taking away detection event (YES in step S1003), then in stepS1004, the CPU 206 determines whether any related object is present invideo images. This is because of waiting until all related objects exit.If any related object is present in video images (YES in step S1004),the CPU 206 continues processing. If there is no related object (NO instep S1004), then in step S1005, the CPU 206 generates a video index. Ifthe operation event is other than a desertion/taking away detectionevent (NO in step S1003), then in step S1010, the CPU 206 determineswhether the detection event is a playback instruction input. This occursas a result of an event on the time line being designated. If thedetection event is the playback instruction input (YES in step S1010),then in step S1011, the CPU 206 plays back recorded video imagesaccording to the index. If the detection event is not the playbackinstruction input (NO in step S1010), then in step S1020, the CPU 206performs other event processing before returning.

According to the processing described above, when an event occurs, avideo index in which only important scenes are edited is generated afterwaiting until related objects exit. More specifically, when a takingaway detection event occurs, an index including the entry of a personwho would take a taking-away act, the moment of taking away, and up tothe exit of the person who has taken a taking-away act is generated toplay back video images.

In a third exemplary embodiment of the present invention, processing tomake a correspondence relation between the object region display and theevent display on a time line intuitively understandable in a monitoringsystem having the time line will be described.

The method for indicating a correspondence relation between the objectregion display and the event display on the time line in the presentexemplary embodiment will be described with reference to FIG. 12. FIG.12 is a diagram illustrating an example of a correspondence relationbetween the object region display and the event display on the timeline. In FIG. 12, an image display region 310 and a frame 312 indicatingan object region in a camera are in a selected state. This state occursby the frame 312 indicating an object region being clicked using a mousepointer by a user. At this point, events of the object related to theframe 312 are highlighted on the time line. Events of an object are, forexample, the appearance and exit of the object. In the event informationillustrated in FIG. 4, the playback apparatus 200 can identifycorrespondence relations based on the same object ID. At this point, theplayback apparatus 200 can make correspondences between objects andevents clear by highlighting, like the frame 312 indicating an objectregion, the frame 312 indicating an object region and events 342, 343,and 344 related to the object. More specifically, the playback apparatus200 uniforms the frame thickness, line type, color, and the like.Accordingly, the user can easily identify only events corresponding to aspecified object from among many events on the time line.

The playback apparatus 200 can change the color of the frame 312indicating an object region in a video image and the display of theevent 342 on the time line according to changes in attributes of theobject. Assume, for example, that the display of the event 342 is anappearance event of the object. Here, a desertion detection event isdefined as an event and it is assumed that when the object is in astopped state for a predetermined time or longer, the event isgenerated. In such a case, the playback apparatus 200 can indicate thatan occurrence of the event is approaching by changing the color of theframe 312 like blue-green-yellow-red according to a stop time of theobject. In this case, by also changing the color of the display of theevent 342 on the time line 340 in conjunction with the frame 312indicating an object region, the playback apparatus 200 can maintain thecorrespondence relation. The playback apparatus 200 indicates thecorrespondence relation between an object and the event display and canalso indicate stop time information of the object only by the eventdisplay on the time line.

The playback apparatus 200 in the present exemplary embodimenthighlights, when an event is specified, events concerning the sameobject and can also highlight corresponding video images and the objectdisplay. This means that if the user selects the display of, forexample, the event 342, the playback apparatus 200 highlights thedisplay of the events 342, 343, and 344 concerning the same object andalso highlights the corresponding image display region 310 and the frame312 indicating the object region. If the user specifies the display ofthe event 342 by double-clicking the mouse in this state, the playbackapparatus 200 starts the playback of video images of the correspondingtime.

Another example of emphasizing events concerning the same object on thetime line is illustrated in FIG. 13. FIG. 13 is a diagram illustratinganother example emphasizing events concerning the same object on thetime line. In FIG. 13, like the time line in FIG. 12, the playbackapparatus 200 highlights events concerning the same object and furthercolors a background 350 between an appearance event of an object and anexit event separately from the normal background. The playback apparatus200 may display, as illustrated in FIG. 13, events concerning otherobjects than the object selected by the user simultaneously or keep suchevents hidden behind.

Incidentally, the playback apparatus 200 can establish correspondencerelations between the object display and the event display even if thereis a plurality of cameras. In other words, the playback apparatus 200displays the object so as to make clear all the correspondence relationsbetween the object and the event among the plurality of cameras, if thesame object is taken by the plurality of cameras disposed at differentpositions.

FIG. 14 is a diagram illustrating an example of the screen of arecording application when objects and events among a plurality ofcameras are displayed by establishing correspondences therebetween. InFIG. 14, it is assumed that the frame 312 indicating an object regionpresent in the image display region 310 and an object 321 present in thecamera video image 320 are the same object. In FIG. 14, there is aplurality of methods for the playback apparatus 200 to determine whetherobjects in a plurality of camera video images are the same object. Ifthere are events of the same type occurring at the same time in aplurality of video images, the events can be considered to concern thesame object. This means that if desertion detection occurs at the sametime in different video images, the playback apparatus 200 considers theobjects to be the same. A method for verifying the identity of objectsin scenes by the playback apparatus 200 based on the physicalrelationship of a plurality of cameras is known as another method.

If the user selects the frame 312 indicating an object region in theimage display region 310 of the network camera 101, the frame 312 ishighlighted by the playback apparatus 200. At the same time, eventinformation is referenced by the playback apparatus 200 to highlightevents 345, 347, and 348 related to the object on the time line 340.Next, the playback apparatus 200 checks whether there is any applicableevent concerning the same object and, if there is any, highlights suchan event. In FIG. 14, an event 346 is highlighted. It is assumed herethat a rule that events of the same type occurring at the same time areconsidered to concern the same object is used. In the example in FIG.14, desertion detection events occur at the same time in the imagedisplay region 310 of the network camera 101 and the camera video image320 of the camera 102. To show that desertion detection events hasoccurred at the same time, the playback apparatus 200 displays aplurality of events by shifting the position. Further, the playbackapparatus 200 highlights the object 321 corresponding to the event 346in the same manner. With the above processing, the playback apparatus200 can display the same object 321 in video images different fromevents related to the frame 312 indicating an object region byestablishing a correspondence therebetween.

At this point, the playback apparatus 200 also adds correspondencesbetween objects to the event management in FIG. 4. For example, theplayback apparatus 200 considers a standing-still detection ON event(for example, the device ID: Cam02, object ID: 003) that has occurred ina different camera at the same time as a standing-still detection ONevent (event ID: 00128, device ID: Cam01, object ID: 002) in FIG. 4 asevents concerning the same object. Then, the playback apparatus 200stores the object ID: 002 and the object ID: 003 as the same object.

Next, the processing procedure for one process by the playback apparatus200 in the present exemplary embodiment will be described with referenceto FIG. 15. FIG. 15 is a flow chart illustrating an example of theprocessing procedure of one process by the playback apparatus 200. Thisprocess is a process concerning the GUI and more specifically, an eventprocessing process. Like FIGS. 8 and 11, for simplification ofdescription, only events directly related to the present exemplaryembodiment are illustrated.

In step S1400, the CPU 206 initializes the display. In step S1401, theCPU 206 waits for an event. Next, in step S1402, the CPU 206 determineswhether any video frame is specified. This determines whether a specificcameral video frame is specified by the user using a mouse pointer. If avideo frame is specified (YES in step S1402), the processing proceeds tostep S1404. In step S1404, the CPU 206 highlights the specified frameand, then in step S1405, highlights events related to the specifiedcamera.

If the event is not an event to specify a video frame (NO in stepS1402), then in step S1410, the CPU 206 determines whether the event isan event to specify an object. If the event is an event to specify anobject (YES in step S14102), then in step S1411, the CPU 206 detects thespecified range. This is because a plurality of objects may bespecified. Next, in step S1412, the CPU 206 highlights events concerningthe same object. In step S1413, the CPU 206 highlights objects relatedto the specified event. If the event is not an event to specify anobject (NO in step S1410), then in step S1420, the CPU 206 determineswhether the event is an event to specify an event. If the event is anevent to specify an event (YES in step S1420), then in step S1421, theCPU 206 detects the specified range. This is because a plurality ofevents may be specified. In step S1422, the CPU 206 highlights eventsconcerning the same object. In step S1423, the CPU 206 highlightsobjects related to the specified event. If the event is not an event tospecify an event (NO in step S1420), then in step S1430, the CPU 206performs other event processing.

According to the present exemplary embodiment, as described above,correspondence relations between objects and events can be made clear byuniforming a highlighting method of the object display in video imagesand the event display on the time line. Also according to the presentexemplary embodiment, a display method that makes correspondencesbetween objects and events in a plurality of camera video images easilyunderstandable can be realized for an event display form in which allevents occurring in different camera video images are displayed on onetime line.

According to each of the above exemplary embodiments, when an eventoccurs, the playback apparatus 200 plays back short video imagesincluding before and after the appearance time of a parent object of anobject related to the event and the time when the object separates fromthe parent object. Accordingly, a user can check video images mostdesired to know concerning the event. Moreover, with only eventscorresponding to the object specified by the user being highlighted bythe playback apparatus 200, correspondence relations between the objectdisplay in a plurality of video images and the event display on the timeline becomes clear. Further, changes in object attributes can be knownon the time line by both the object display and the event display beingchanged by the playback apparatus 200 according to changes in objectattributes.

Therefore, when an event concerning an object occurs, suitable videoimages concerning objects related to the object can swiftly be playedback.

Aspects of the present invention can also be realized by a computer of asystem or apparatus (or devices such as a CPU or MPU) that reads out andexecutes a program recorded on a memory device to perform the functionsof the above-described embodiment (s), and by a method, the steps ofwhich are performed by a computer of a system or apparatus by, forexample, reading out and executing a program recorded on a memory deviceto perform the functions of the above-described embodiment (s). For thispurpose, the program is provided to the computer for example via anetwork or from a recording medium of various types serving as thememory device (e.g., computer-readable medium).

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all modifications, equivalent structures, and functions.

This application claims priority from Japanese Patent Application No.2009-105560 filed Apr. 23, 2009, which is hereby incorporated byreference herein in its entirety.

1. An apparatus comprising: a recording unit configured to record avideo image; a generation unit configured to generate, when a first typeof event in which a state of an object changes is detected from thevideo image, an index associating the first type and a second type ofevent related to the first type in the video image; and a playback unitconfigured to play back, when the first type in the video image isspecified, a video image concerning the second type corresponding to thespecified first type based on the index.
 2. The apparatus according toclaim 1, wherein, when the first type is detected, the generation unitis configured to generate an index including time concerning the firsttype and time concerning the second type related to the first type. 3.The apparatus according to claim 1, wherein the generation unit isconfigured to detect desertion or taking away of an object as the firsttype and to generate an index associating the desertion or taking awayof the object and an entry or exit of a person who has carried out thedesertion or taking away as the second type related to the first type.4. The apparatus according to claim 1, further comprising: a displayunit configured to display a time line on which symbols indicating thefirst type are arranged along a time axis; and a specifying unitconfigured to specify an event of the first type by selecting the symbolfrom the time line.
 5. The apparatus according to claim 4, wherein thedisplay unit is configured to display the video image concerning theevent detected from the video image in a window together with the timeline.
 6. The apparatus according to claim 5, wherein the display unit isconfigured to associate and display the video image concerning the eventdetected from the video image and the symbol on the time linerepresenting the event.
 7. A method comprising: recording a video image;generating, when a first type of event in which a state of an objectchanges is detected from the video image, an index associating the firsttype and a second type of event related to the first type in the videoimage; and playing back, when the first type of event in the video imageis specified, the video image concerning the second type correspondingto the specified first type based on the index.
 8. The method accordingto claim 7, further comprising: when the first type is detected,generating an index including time concerning the first type and timeconcerning the second type related to the first type.
 9. The methodaccording to claim 7, further comprising: detecting desertion or takingaway of an object as the first type; and generating an index associatingthe desertion or taking away of the object and an entry or exit of aperson who has carried out the desertion or taking away as the secondtype related to the first type.
 10. The method according to claim 7,further comprising: displaying a time line on which symbols indicatingthe first type are arranged along a time axis; and specifying an eventof the first type by selecting the symbol from the time line.
 11. Themethod according to claim 10, further comprising displaying the videoimage concerning the event detected from the video image in a windowtogether with the time line.
 12. The apparatus according to claim 11,further comprising associating and displaying the video image concerningthe event detected from the video image and the symbol on the time linerepresenting the event.
 13. A computer-readable storage medium storing aprogram for causing a computer to execute the video playback methodaccording to claim
 7. 14. The computer-readable storage medium accordingto claim 13, further comprising: when the first type is detected,generating an index including time concerning the first type and timeconcerning the second type related to the first type.
 15. Thecomputer-readable storage medium according to claim 14, furthercomprising: detecting desertion or taking away of an object as the firsttype; and generating an index associating the desertion or taking awayof the object and an entry or exit of a person who has carried out thedesertion or taking away as the second type related to the first type.16. The computer-readable storage medium according to claim 14, furthercomprising: displaying a time line on which symbols indicating the firsttype are arranged along a time axis; and specifying an event of thefirst type by selecting the symbol from the time line.
 17. Thecomputer-readable storage medium according to claim 16, furthercomprising displaying the video image concerning the event detected fromthe video image in a window together with the time line.
 18. Thecomputer-readable storage medium according to claim 17, furthercomprising associating and displaying the video image concerning theevent detected from the video image and the symbol on the time linerepresenting the event.